Methods and apparatus for image analysis using profile weighted intensity features

ABSTRACT

A new morphological feature referred to herein as profile weighted intensity feature is provided, useful for automated classification of objects, such as cells, that are depicted in digital images. In certain embodiments, the profile weighted intensity feature is determined by automatically identifying a border of a cell in an input image, determining a distance image for the cell, computing a profile function for the cell, and computing a mean intensity of at least a portion of the input image weighted by the profile function for the cell.

TECHNICAL FIELD

This invention relates generally to image processing techniques foridentifying and characterizing objects within digital images. Moreparticularly, in certain embodiments, the invention relates to methodsand apparatus for characterizing a new morphological feature referred toherein as a profile weighted intensity feature, of cells in a digitalimage, and applying this feature in the automated classification of cellphenotype, for example, in image analysis software.

BACKGROUND

The ability to automatically classify objects into categories ofinterest has applications across a wide range of industries andscientific fields, including biology, social sciences, and finance. Oneparticular application of interest is the classification of biologicalcells according to cell phenotype.

An accurate and efficient automated cell phenotype classification methodrequires identifying morphological characteristics of individual cells,as pictured in a digital image, which are useful for distinguishingdifferent cell phenotypes. Thus, when using image processing techniquesto perform cell phenotype classification, it is desired to identifymorphological features that vary according to the different cell typesin an image and are characteristic of those cell types. A cell typehaving a unique size, for example, may be identified by evaluating thesizes of the cells in the image. Likewise, a cell type having aparticular characteristic shape or color may be identified by evaluatingthe shapes and colors of the cells in the image. The more amorphological feature (e.g., size, shape, or color) varies from one celltype to the next, the more useful that feature is for distinguishingdifferent types of cells during cell phenotype classification.

Automated image processing techniques are useful to standardize theclassification process for improved accuracy and speed of cellclassification. However, existing automated image processing techniquesare often incapable of distinguishing among the different cellphenotypes in an image. Existing image processing techniques can also beoverly complicated, difficult to describe and implement, andcomputationally intensive.

There is a need for more accurate and efficient image processingtechniques for identifying different types of objects in an image. Inparticular, there is a need for new morphological features that may beused to characterize cells in an image, for the purpose of automatedcell phenotype classification.

SUMMARY OF THE INVENTION

The methods and apparatus described herein are capable of robust andefficient identification and characterization of morphological featuresof objects within an image. A new family of morphological features,referred to herein as profile weighted intensity features, is provided.These profile weighted intensity features are a type of weighted mean orsum intensity of one or more objects (e.g., cells) in an image.

By utilizing these new morphological features, a computationallyefficient and more accurate tool is provided for identifying andclassifying objects in images. For example, it has been discovered thatautomated cell phenotype characterization is improved by determining amean intensity that more heavily weights pixel intensity near theborders of the cell (e.g., the outer cell border and/or the border ofthe cell nucleus), and using this mean intensity value in the algorithmto characterize cell phenotype. It has also been discovered thatrelative (e.g., normalized) intensities may be more favorable thanabsolute intensities for cell classification because large cell-to-cellvariations in cellular samples are cancelled out or reduced when dividedby another intensity of the same cell. In one embodiment, the profileweighted intensity features described herein are normalized. Forexample, for a given cell, a weighted mean intensity may be divided by acorresponding unweighted mean intensity, yielding a relative (unitless)feature. Furthermore, advances in the efficiency of certaincomputational step—for example, the use of a new sliding parabolaerosion operation in the determination of a distance image—facilitatethe efficient use of this new class of morphological features in theclassification of objects (e.g., cells) in a digital image.

In one aspect, the invention relates to a method for determining aprofile weighted intensity feature for a cell useful for classifyingcell phenotype, the profile weighted intensity feature determined fromone or more images of the cell. The method includes the steps of: (a)detecting a cell in an image; (b) computing a profile image for thecell, wherein pixel intensity of the profile image is a function of theclosest distance from the pixel to a border of the cell; and (c)computing a profile weighted intensity feature for the cell using theprofile image of the cell as a weight image, wherein the profileweighted intensity feature for the cell is a weighted mean intensity ofpixels of the cell.

In certain embodiments, the method includes the step of classifying cellphenotype using the computed profile weighted intensity feature. Theborder may be, for example, an outer border of the cell. In oneembodiment, the profile image is computed such that pixels near theborder of the cell are emphasized and pixels a given distance away fromthe border of the cell are deemphasized. In certain embodiments, thedistance image is determined via sliding parabola erosion operationperformed on a cell mask image. In one embodiment, the calculatedfeatures include at least one of unnormalized intensities and normalizedintensities. In certain embodiments, step (c) includes computing aprofile weighted intensity feature for the cell from a filtered image ofan acquired input image of the cell using the profile image of the cellas a weight image.

In another aspect, the invention relates to a method for determining aprofile weighted intensity feature for a cell useful for classifyingcell phenotype, the profile weighted intensity feature determined fromone or more images of the cell. The method includes the steps of: (a)detecting a cell and detecting a nucleus of the cell from one or moreimages of the cell; (b) computing a profile image for the cell, whereinpixel intensity of the profile image is a function of both (i) theclosest distance from the pixel to an outer border of the cell and (ii)the closest distance from the pixel to an outer border of a nucleus ofthe cell; and (c) computing a profile weighted intensity feature for thecell using the profile image of the cell as a weight image, wherein theprofile weighted intensity feature for the cell is a weighted meanintensity of pixels of the cell. In certain embodiments, the profileweighted intensity feature for the cell is computed from a filteredimage of an acquired input image of the cell using the profile image ofthe cell as a weight image.

In another aspect, the invention relates to a method for determining aprofile weighted intensity feature for a cell useful for classifyingcell phenotype, the profile weighted intensity feature determined fromone or more images of the cell. The method includes the steps of: (a)detecting a plurality of borders of a cell from one or more images ofthe cell; (b) computing a profile image for the cell, wherein pixelintensity of the profile image is a function of the nearest distancesfrom the pixel to each of the plurality of detected borders of the cell;and (c) computing a profile weighted intensity feature for the cellusing the profile image of the cell as a weight image, wherein theprofile weighted intensity feature for the cell is a weighted meanintensity of pixels of the cell. In certain embodiments, the profileweighted intensity feature for the cell is computed from a filteredimage of an acquired input image of the cell using the profile image ofthe cell as a weight image.

In another aspect, the invention relates to an apparatus fordetermination of a profile weighted intensity feature for a cell usefulfor classifying cell phenotype, the profile weighted intensity featuredetermined from one or more images of the cell. The apparatus includes amemory for storing a code defining a set of instructions, and aprocessor for executing the set of instructions. The code includes aprofile weighted intensity module configured to: (i) detect a cell in animage; (ii) compute a profile image for the cell, wherein pixelintensity of the profile image is a function of the closest distancefrom the pixel to a border of the cell (e.g., an outer border of thecell); and (iii) compute a profile weighted intensity feature for thecell using the profile image of the cell as a weight image, wherein theprofile weighted intensity feature for the cell is a weighted meanintensity of pixels of the cell.

In certain embodiments, the profile weighted intensity module isconfigured to classify cell phenotype using the computed profileweighted intensity feature. In certain embodiments, the profile image iscomputed such that pixels near the border of the cell are emphasized andpixels a given distance away from the border of the cell aredeemphasized. In certain embodiments, the distance image is determinedvia sliding parabola erosion operation performed on a cell mask image.

Elements of embodiments described with respect to a given aspect of theinvention may be used in various embodiments of another aspect of theinvention. For example, it is contemplated that features of dependentclaims depending from one independent claim can be used in apparatusand/or methods of any of the other independent claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The objects and features of the invention can be better understood withreference to the drawing described below, and the claims.

FIG. 1 is an image of a cell acquired using a detector that is sensitiveto light emitted by a specific fluorophore, according to an illustrativeembodiment of the present invention.

FIG. 2 is an image of the cell of FIG. 1 acquired using a detector thatis sensitive to a nucleus stain, according to an illustrative embodimentof the present invention.

FIG. 3 is an image of the cell of FIG. 1 in which the cell and the cellnucleus have been identified, according to an illustrative embodiment ofthe present invention.

FIG. 4 is a distance image for the cell of FIG. 1 in which pixelintensity is a function of distance from an outer border of the cell,according to an illustrative embodiment of the present invention.

FIG. 5 is a profile image calculated from the distance image of FIG. 4,according to an illustrative embodiment of the present invention.

FIG. 6 is a distance image for the cell of FIG. 1 in which pixelintensity is a function of distance from a nucleus border, according toan illustrative embodiment of the present invention.

FIG. 7 is a profile image calculated from the distance image of FIG. 6,according to an illustrative embodiment of the present invention.

FIG. 8 is an image obtained by applying a texture energy filter to theimage of FIG. 1, according to an illustrative embodiment of the presentinvention.

FIG. 9A is a flow chart of a first example method for determining of aprofile weighted intensity features for a cell useful for classifyingcell phenotype, according to an illustrative embodiment of the presentinvention.

FIG. 9B is a flow chart of a second example method for determination ofa profile weighted intensity feature for a cell useful for classifyingcell phenotype, according to a illustrative embodiment of the presentinvention.

FIG. 9C is a flow chart of a third example method for determination of aprofile weighted intensity feature for a cell useful for classifyingcell phenotype, according to an illustrative embodiment of the presentinvention.

FIGS. 10 a and 10 b include images from two different samples of acell-based assay representing two different cell classes, according toan illustrative embodiment of the present invention.

FIGS. 11 a and 11 b include images from the same cell-based assaydepicted in FIGS. 10 a and 10 b, respectively, with the detected cellsmasked (i.e., pixels outside the cells are black) and cell and nucleusborders highlighted, according to an illustrative embodiment of thepresent invention.

FIG. 12 is an x,y-plot of a best pair of features for separating the twoclasses of objects depicted in FIGS. 11 a and 11 b, according to anillustrative embodiment of the present invention.

FIG. 13 is a block diagram of an apparatus for determination of aprofile weighted intensity feature for a cell useful for classifyingcell phenotype, according to an illustrative embodiment of the presentinvention.

DESCRIPTION

It is contemplated that apparatus, systems, methods, and processes ofthe claimed invention encompass variations and adaptations developedusing information from the embodiments described herein. Adaptationand/or modification of the apparatus, systems, methods, and processesdescribed herein may be performed by those of ordinary skill in therelevant art.

Throughout the description, where systems are described as having,including, or comprising specific components, or where processes andmethods are described as having, including, or comprising specificsteps, it is contemplated that, additionally, there are systems of thepresent invention that consist essentially of, or consist of, therecited components, and that there are processes and methods accordingto the present invention that consist essentially of, or consist of, therecited processing steps.

It should be understood that the order of steps or order for performingcertain actions is immaterial so long as the invention remains operable.Moreover, two or more steps or actions may be conducted simultaneously.

The mention herein of any publication, for example, in the Backgroundsection, is not an admission that the publication serves as prior artwith respect to any of the claims presented herein. The Backgroundsection is presented for purposes of clarity and is not meant as adescription of prior art with respect to any claim.

When using image processing techniques, a cell phenotype classificationprocess begins with the acquisition of an image or images depicting oneor more cells. The next step is typically a segmentation step to detectone or more cells or portions of cells (e.g., a cell nucleus) in theimage or images. Next, a set of features is extracted from theidentified cells and used to characterize and classify the cells. Incertain embodiments, one or more image filters are applied as part ofthe feature extraction step. The filters may include one or more textureenergy filters, which are a special subset of image filters withnon-negative output. In certain embodiments, the same filter that isused for extraction of texture features is additionally used forextraction of morphology features. In one embodiment, unweighted meanintensity of a texture-filtered image is a texture feature, and profileweighted mean intensity of the same texture-filtered image is amorphology feature.

Among the different types of images, a distance image is an image inwhich pixel intensity is proportional to a distance between the pixeland some other location in the image. For example, in a distance imageof a biological cell, the intensity at each pixel within the cell maydepend on the distance between the pixel and an edge of the cell, suchas the outer edge of the cell or the edge of the nucleus. Therelationship between intensity and distance may be linear or non-linear.

Weighted sum calculation and weighted mean calculation are examples ofimage processing techniques. Weighted sum calculation is determined byapplying weights to the pixel intensities in an image (e.g., usinganother image or a set of data to provide the weights) and then addingthe weighted pixel intensities. Similarly, weighted mean calculation isdetermined by applying weights to the pixel intensities in an image andthen averaging the weighted pixel intensities. The resulting valueobtained for both weighted sum calculation and weighted mean calculationis influenced more by some pixels (i.e., those that are weightedheavily) and less by other pixels (i.e., those that are not weightedheavily).

The methods and apparatus described herein provide a new family ofmorphological features for characterizing and/or classifying one or moreobjects (e.g., biological cells) in an input image. In one embodiment,the new family of morphological features is obtained by applying afilter (e.g., a texture energy filter) to the image. A distance image iscalculated in which pixel intensity is a function of distance to theclosest border pixel. For example, the distance image is advantageouslycalculated using the sliding parabola erosion technique described inU.S. patent application Ser. No. 13/230,377, filed Sep. 12, 2011,entitled “Methods and Apparatus for Fast Identification of RelevantFeatures for Classification or Regression,” the disclosure of which ishereby incorporated by reference herein in its entirety. The profileimage is then used as a weight image for calculating the mean or sumintensity of another image, such as the input image.

FIGS. 1 through 8 are images of a cell 100, in accordance with certainembodiments of the invention. FIG. 1 is an image of the cell 100acquired with a detector that is sensitive to light emitted by aspecific fluorophore. Morphological features (e.g., numericalproperties) within the cell 100 are indicated by variations in intensitywithin the image. The intensity variations may be quantified to identifyand characterize morphological features of interest.

FIG. 2 is another image of the cell 100 acquired with a detector that issensitive to a nucleus stain. This type of image may be useful fornucleus detection. For example, FIG. 3 is a similar image of the cell100 after the cell 100, a cell nucleus 302, and cytoplasm 304 have beenidentified during segmentation. Pixels not belonging to the cell 100 areblack. Pixels belonging to a border 306 of the nucleus 302 arehighlighted. In certain embodiments, one input image is acquired fornuclei detection and another image is used for feature calculation ordetection. In one embodiment, more than one image may be used forcalculation of features.

FIG. 4 is a distance image for the cell 100 in which pixels outside ofthe cell 100 are black and each pixel within the cell 100 has anintensity that is equal to or a function of (e.g., a linear function, apolynomial function, an exponential function, and/or a trigonometricfunction) the distance between the pixel and an outer border 402 of thecell 100. FIG. 5 is a profile image for the cell 100 calculated from thedistance image of FIG. 4. The profile image characterizes or emphasizesthe part of cytoplasm 304 closest to the outer border 402 (e.g., a cellmembrane).

FIG. 6 is another distance image for the cell 100 in which each pixelwithin cytoplasm 304 of the cell 100 has an intensity equal to afunction of the distance from the nucleus border 306. Pixels outside ofthe cell 100 and pixels within the nucleus 302 are black. FIG. 7 is aprofile image for the cell 100 calculated from the distance image ofFIG. 6. The profile image characterizes or emphasizes a part ofcytoplasm 304 closest to the nucleus 302.

In certain embodiments, a profile image is a function of a singledistance, even if there is more than one border (e.g., the nucleusborder 306 and the outer border 402), as shown in FIGS. 5 and 7. Inalternative embodiments, the profile image is equal to or a function oftwo distances. For example, a distance image may be generated in whichthe intensity of the distance image is a function of distance from anucleus border and an outer cell border. A profile image may then begenerated from the distance image such that the intensities in theprofile image are a function of the distances from the two borders. Theessence of the step of calculating a profile image does not depend onthe number of borders considered for the calculation.

In certain embodiments, more than one profile image is calculated from agiven distance image. For example, each profile image may be obtainedusing a different mathematical relationship between pixel intensity inthe distance image and pixel intensity in the profile image.

FIG. 8 is an image of the cell 100 obtained by applying a texture energyfilter to the image of FIG. 1. As depicted, the texture energy filterdetects and visualizes ridge-like structures 802 within the cell 100. Incertain embodiments, morphological features characterizing the cell 100are derived from this image in addition to or instead of the image ofFIG. 1.

As mentioned above with respect to FIGS. 5 and 7, in certainembodiments, an object includes more than one border. For example, acell may have a border at its outer edge (i.e., a cell border) and aborder at the edge of its nucleus (i.e., a nucleus border). When twoborders are identified for a single cell, there are two distances toconsider for calculation of the distance image and/or the profile image.For example, the intensity of a pixel within the cell may be a functionof the distance to the cell border and the distance to the nucleusborder. In other words, intensities with the profile function (i.e., thefunction used to calculate intensities within a cell in a profile image)may be defined as a function of both distances in such case.

It is a convention that distances inside and outside the border are ofopposite sign, considered as a signed coordinate. From the viewpoint ofthe present methods and apparatus, this convention may be useful. Forexample, the profile function must be defined on both sides of theborder between nucleus and cytoplasm.

A profile function is a function of a distance, or distances if thereare many borders. For example, the profile function may beprofile=exp(−distance*distance/(2*w²)), where profile is pixel intensitywithin a cell of the profile image, distance is the distance from apixel to a border (or from a pixel to more than one border), and w is awidth parameter. In certain embodiments, w is the width of the regionnear the border that will be characterized by the given intensityprofile feature. The width w may be specified by a user and/or may havea default value. In this particular example, the profile function ishigh near the border of the cell and low when distance to the border ismuch higher than parameter w. The profile image must be calculatedindividually for each cell since each cell has an individual shape ofthe border line.

After the profile function has been used to calculate pixel intensitieswithin the profile image, the profile image is used as weight functionto calculate weighted average intensity of the input image for eachcell. With this feature, each cell is characterized by a single numericattribute, which may be referred to as the profile weighted intensityfeature. When more than one input image and/or more than one profileimage have been obtained, the profile weighted intensity feature may becalculated for each combination of input and profile images.

FIG. 9A through 9C are flowcharts of example methods for determiningprofile weighted intensity features of cells useful for classifying cellphenotype, in accordance with various embodiments of the invention. Thefeatures are determined from one or more images of the cells. Turning toFIG. 9A, at step 902, the method 900 includes detecting a cell in animage. At step 904, the method 900 includes computing a profile imagefor the cell, wherein pixel intensity of the profile image is a functionof the closest distance from the pixel to a border of the cell.According to one embodiment, the border is the outer border of the cellsuch as the outer border 402 of the cell 100 of FIG. 4. The border maybe, for example, a cell border and/or a nucleus border. In oneembodiment, the border is identified by applying an image filter and/ormask to the input image. In certain embodiments, the profile image iscomputed such that pixels near the border of the cell are emphasized andpixels a given distance away from the border of the cell aredeemphasized as illustrated by step 912 of the method 910 of FIG. 9B. Atstep 906, the method 900 includes computing a profile weighted intensityfeature for the cell using the profile image of the cell as a weightimage, wherein the profile weighted intensity feature for the cell is aweighted mean intensity. The method 900 may also include step 908 ofclassifying cell phenotype using at least the computed profile weightedintensity.

In certain embodiments, the method 920 of FIG. 9C determines a distanceimage by performing a sliding parabola erosion operation on a mask imageof the cell as illustrated by step 922 of FIG. 9C, thereby producing aneroded mask image that is related to or a known function of the distanceimage. The parabola-eroded mask image may then be used to determine orreconstruct the profile image. However, the profile image is notnecessarily equal to or proportional to the distance image. Otherfunctions of the distance image may be utilized to determine the profileimage, for example exponential, Gaussian, or step functions, asidentified by step 924 of FIG. 9C.

While sliding parabola erosion may be a preferable method fordetermining the distance image, other operations or techniques may alsobe used. For example, the distance image may be determined using rollingball erosion and/or binary mask erosion/dilation operations, instead ofsliding parabola erosion. In certain embodiments, the distance image isdetermined by calculating the distance between each pixel and one ormore border. In one embodiment, sliding parabola erosion is a preferredtechnique because it is inexpensive (i.e., not computationallyintensive) and accurate.

The method 900 may also include calculating a profile weighted intensityfor the cell. When calculating a weighted mean intensity or intensities,image intensity is weighted by one or more profile functions, eachcalculated from one or more distance image.

Alternatively, in certain embodiments, the pixel intensity of a profileimage is a function of both (i) the closest distance from the pixel toan outer border of the cell and (ii) the closest distance from the pixelto an outer border of a nucleus of the cell. In another embodiment, thepixel intensity of a profile image is a function of the nearestdistances from the pixel to each of a plurality of detected borders ofthe cell. When used for cell phenotype classification, the profileweighted intensity feature for the cell may be calculated using theprofile image of the cell as a weight image.

In certain embodiments, calculation of the profile weighted intensityinvolves (i) filtering the image, (ii) calculating a distance imageR(x,y), or a series of distance images R₁(x,y), R₂(x,y), . . . ,R_(N)(x,y), and (iii) calculating a profile image using a profilefunction W(R) or W(R₁, R₂, . . . , R_(N)), or a series of profilefunctions W₁(R), W₂(R), . . . , W_(M)(R), or W₁(R₁,R₂, . . . , R_(N)),W₂(R₁, R₂, . . . , R_(N)), . . . , W_(M)(R₁, R₂, . . . , R_(N)).

Calculation of a distance image R(x,y) may take into consideration oneor more borders, such as the cell border and the nucleus border. In thecase of an internal border (e.g., the nucleus border between a nucleusand a cytoplasm), the distance from border may be measured in twodifferent directions, with one of the directions considered to be anegative direction. As mentioned, sliding parabola erosion may be usedto calculate a distance image R(x,y).

Calculation of a profile function, or a series of profile functions,also takes into consideration the distance from one or more borders. Ifthere is more than one distance function (e.g., a function for thedistance from cell border and a function for the distance from thenucleus border), then the set of profile functions may be diversified.In one embodiment, the profile function is a function of the distance,R, i.e., W=W(R). The profile image is thenW(x, y)=W(R(x, y)).

When calculating a feature characterizing an object, in certainembodiments, a non-normalized feature is mean intensity of an inputimage weighted by a profile function. For example, if the profile imageis denoted by W and the input image is denoted by I, then thenon-normalized profile feature F_(u) is expressed as

$F_{u} = {\frac{\int{{W\left( {x,y} \right)}{l\left( {x,y} \right)}{\mathbb{d}x}{\mathbb{d}y}}}{\int{{W\left( {x,y} \right)}{\mathbb{d}x}{\mathbb{d}y}}}.}$In some embodiments, it is desirable to normalize the profile feature bydividing it by another intensity. This normalized profile feature F_(u)is expressed as

$F_{n} = {\frac{\int{{W\left( {x,y} \right)}{l\left( {x,y} \right)}{\mathbb{d}x}{\mathbb{d}y}{\int{{\mathbb{d}x}{\mathbb{d}y}}}}}{\int{{W\left( {x,y} \right)}{\mathbb{d}x}{\mathbb{d}y}{\int{{l\left( {x,y} \right)}{\mathbb{d}x}{\mathbb{d}y}}}}}.}$

Filtering the input image is an optional step. The filtering may beperformed using one or more texture energy filters, for example.

In certain embodiments, profile images are created using severaldifferent methods. For example, a single distance image may be used tocreate a plurality of profile images. A different mathematical functionmay be used for each profile image to relate intensity in the profileimage to intensity in the distance image. In another approach, profileimages are created from distance images having multiple borders, such asthe distance image in FIG. 6. Compared to a single border, two or moreborders allows a higher diversity of profile images to be created. Thenumber of features may be further diversified by using not only theoriginal image but also a filtered image or even a set of filteredimages. For each filtered image, additional profile images may becalculated and, as described here, used to identify and characterizedmorphological features. In certain embodiments, the calculated featuresmay be unnormalized intensities, or normalized ones.

FIGS. 10 a through 12 illustrate the use of profile weighted intensityfeatures for the classification of cell phenotype, in accordance withcertain embodiments of the invention. FIGS. 10 a and 10 b include a pairof images of cells 1000 from two different samples of a cell-basedassay. FIGS. 11 a and 11 b include images of the same cells 1000 inFIGS. 10 a and 10 b, respectively, but the detected cells are masked(i.e., pixels outside the cells are turned black) and cell borders andnucleus borders are highlighted. Comparing the pair of figures, one maynotice that the cell class depicted in FIG. 11 b includes bright spottyregions 1100 around the nuclei of the cells 1000. These spotty regions1100 are less noticeable and located more randomly in the cells 1000 ofthe other cell class depicted in FIG. 11 a.

FIG. 12 includes an x,y-plot of the best pair of features for separatingthe two classes of objects depicted in FIGS. 11 a and 11 b. The bestpair of features for separating the two classes has been automaticallyselected by the computer among 165 features routinely calculated foreach cell. Each data point represents a cell, with the white and blackcircles corresponding to two classes of cells, each from differentcontrol samples. The x-axis in this figure is a profile weightedintensity feature calculated from a dark-filtered image. This featurecharacterizes texture in a region of cytoplasm near the nucleus. Indeed,in this region, the cells of the two classes differ the most. Thegrouping or separation of the black and white circles in this figureindicates that the weighted profile feature (represented by the x-axis)is useful for distinguishing between different cell phenotypes.

The methods and apparatus described herein are applicable universally todata or images of any dimensionality. For example, the equationspresented above can be easily applied to a 3-D image, in which R and Ware functions of x, y, and z. Because 3-D images generally have morevoxels than 2-D images, calculation times are generally longer for 3-Dimages.

In certain embodiments, the methods described above are implemented on acomputer using computer software. In one specific embodiment, to performfeature extraction, the software includes three building blocks tocalculate intensity properties, calculate texture properties, andcalculate morphology properties. In one embodiment, the softwareimplements a method for extracting many morphology features in parallel,thereby enabling features from several (e.g., five) different familiesto be combined. One of the families is profile weighted intensity. Tocalculate profile weighted intensity features, a user applies thebuilding block for calculating morphology properties. The user thenensures that the family of profile weighted intensity features isselected (by default, it is selected). Next, a user may select one ormore filters (e.g., an energy filter) to apply and specify inputparameters for the filter(s). In another embodiment, a wide set offeatures is calculated automatically whenever the user requests aclassification or regression task. Later, when the tuning is completed(i.e. the relevant features have been identified), only the relevantfeatures will be calculated.

FIG. 13 is block diagram of the apparatus 1300 for determination of aprofile weighted intensity feature for a cell useful for classifyingcell phenotype. The apparatus 1300 includes a memory 1302 for storing acode 1304 defining a set of instructions. The apparatus also includes aprocessor 1306 for executing the set of instructions. The code 1304includes a profile weighted intensity module 1308 configured to executesteps such as those described in FIGS. 9A through 9C above.

Regarding computation times, the most expensive step in the computationsmay be the calculation of the distance image (or more than one distanceimage if different borders are involved). For calculation of a distanceimage, sliding parabola erosion may be employed. In one embodiment, thecalculation time for performing sliding parabola erosion on a 1360×1024pixel image is about 0.17 seconds, using a Dell Latitude 630 (2.2 GHzlaptop). All other operations are relatively cheap (i.e., faster). Inanother embodiment, when a series of profile functions are all based onthe same distance function, the whole set is calculated at little or noadditional cost. By comparison, when there are several distancefunctions in a set, each distance function requires approximately thesame amount of computation time.

It should be noted that embodiments of the present invention may beprovided as one or more computer-readable programs embodied on or in oneor more articles of manufacture. The article of manufacture may be anysuitable hardware apparatus, such as, for example, a floppy disk, a harddisk, a CD ROM, a CD-RW, a CD-R, a DVD ROM, a DVD-RW, a DVD-R, a flashmemory card, a PROM, a RAM, a ROM, or a magnetic tape. In general, thecomputer-readable programs may be implemented in any programminglanguage. Some examples of languages that may be used include C, C++, orJAVA. The software programs may be further translated into machinelanguage or virtual machine instructions and stored in a program file inthat form. The program file may then be stored on or in one or more ofthe articles of manufacture.

A computer hardware apparatus may be used in carrying out any of themethods described herein. The apparatus may include, for example, ageneral purpose computer, an embedded computer, a laptop or desktopcomputer, or any other type of computer that is capable of runningsoftware, issuing suitable control commands, receiving graphical userinput, and recording information. The computer typically includes one ormore central processing units for executing the instructions containedin software code that embraces one or more of the methods describedherein. The software may include one or more modules recorded onmachine-readable media, where the term machine-readable mediaencompasses software, hardwired logic, firmware, object code, and thelike. Additionally, communication buses and I/O ports may be provided tolink any or all of the hardware components together and permitcommunication with other computers and computer networks, including theinternet, as desired. The computer may include a memory or register forstoring data.

In certain embodiments, the modules described herein may be softwarecode or portions of software code. For example, a module may be a singlesubroutine, more than one subroutine, and/or portions of one or moresubroutines. The module may also reside on more than one machine orcomputer. In certain embodiments, a module defines data by creating thedata, receiving the data, and/or providing the data. The module mayreside on a local computer, or may be accessed via network, such as theInternet. Modules may overlap—for example, one module may contain codethat is part of another module, or is a subset of another module.

The computer can be a general purpose computer, such as a commerciallyavailable personal computer that includes a CPU, one or more memories,one or more storage media, one or more output devices, such as adisplay, and one or more input devices, such as a keyboard. The computeroperates using any commercially available operating system, such as anyversion of the Windows™ operating systems from Microsoft Corporation ofRedmond, Wash., or the Linux™ operating system from Red Hat Software ofResearch Triangle Park, N.C. The computer is programmed with softwareincluding commands that, when operating, direct the computer in theperformance of the methods of the invention. Those of skill in theprogramming arts will recognize that some or all of the commands can beprovided in the form of software, in the form of programmable hardwaresuch as flash memory, ROM, or programmable gate arrays (PGAs), in theform of hard-wired circuitry, or in some combination of two or more ofsoftware, programmed hardware, or hard-wired circuitry. Commands thatcontrol the operation of a computer are often grouped into units thatperform a particular action, such as receiving information, processinginformation or data, and providing information to a user. Such a unitcan comprise any number of instructions, from a single command, such asa single machine language instruction, to a plurality of commands, suchas a plurality of lines of code written in a higher level programminglanguage such as C++. Such units of commands are referred to generallyas modules, whether the commands include software, programmed hardware,hard-wired circuitry, or a combination thereof. The computer and/or thesoftware includes modules that accept input from input devices, thatprovide output signals to output devices, and that maintain the orderlyoperation of the computer. The computer also includes at least onemodule that renders images and text on the display. In alternativeembodiments, the computer is a laptop computer, a minicomputer, amainframe computer, an embedded computer, or a handheld computer. Thememory is any conventional memory such as, but not limited to,semiconductor memory, optical memory, or magnetic memory. The storagemedium is any conventional machine-readable storage medium such as, butnot limited to, floppy disk, hard disk, CD-ROM, and/or magnetic tape.The display is any conventional display such as, but not limited to, avideo monitor, a printer, a speaker, an alphanumeric display. The inputdevice is any conventional input device such as, but not limited to, akeyboard, a mouse, a touch screen, a microphone, and/or a remotecontrol. The computer can be a stand-alone computer or interconnectedwith at least one other computer by way of a network. This may be aninternet connection.

As used herein, an “image”—for example, an image of one or morecells—includes any visual representation, such as a photo, a videoframe, streaming video, as well as any electronic, digital ormathematical analogue of a photo, video frame, or streaming video. Anyapparatus described herein, in certain embodiments, includes a displayfor displaying an image or any other result produced by the processor.Any method described herein, in certain embodiments, includes a step ofdisplaying an image or any other result produced via the method.

In certain embodiments, the methods and apparatus described herein areused for cell phenotype classification and may include the featureselection module described in U.S. patent application Ser. No.13/230,377, filed Sep. 12, 2011, entitled “Methods and Apparatus forFast Identification of Relevant Features for Classification orRegression,” the disclosure of which is hereby incorporated by referenceherein in its entirety.

In certain embodiments, the methods and apparatus described hereinutilize sliding parabola erosion. A sliding parabola erosion procedureis described in U.S. patent application Ser. No. 13/230,433, filed Sep.12, 2011 entitled “Methods and Apparatus for Image Analysis andModification Using Fast Sliding Parabola Erosion,” the disclosure ofwhich is hereby incorporated by reference herein in its entirety.

EQUIVALENTS

While the invention has been particularly shown and described withreference to specific preferred embodiments, it should be understood bythose skilled in the art that various changes in form and detail may bemade therein without departing from the spirit and scope of theinvention as defined by the appended claims.

What is claimed is:
 1. A method for determining a profile weightedintensity feature for a cell useful for classifying cell phenotype, saidprofile weighted intensity feature determined from one or more images ofthe cell, the method comprising the steps of: (a) detecting, by aprocessor of a computer, a cell in an image; (b) computing, by theprocessor, a profile image for the cell, wherein pixel intensity of theprofile image is a function of the closest distance from the pixel to aborder of the cell; and (c) computing, by the processor, a profileweighted intensity feature for the cell using the profile image of thecell as a weight image, wherein the profile weighted intensity featurefor the cell is a weighted mean intensity of pixels of the cell.
 2. Themethod of claim 1, further comprising the step of classifying cellphenotype using the computed profile weighted intensity feature.
 3. Themethod of claim 1, wherein the border is an outer border of the cell. 4.The method of claim 1, wherein the profile image is computed such thatpixels near the border of the cell are emphasized and pixels a givendistance away from the border of the cell are deemphasized.
 5. Themethod of claim 1, comprising computing a distance image is via slidingparabola erosion operation performed on a cell mask image, wherein theprofile image is based at least in part on the distance image.
 6. Themethod of claim 1, wherein the profile weighted intensity featureincludes at least one of unnormalized intensities and normalizedintensities.
 7. The method of claim 1, wherein step (c) comprisescomputing the profile weighted intensity feature for the cell from afiltered image of an acquired input image of the cell using the profileimage of the cell as the weight image.
 8. A method for determining aprofile weighted intensity feature for a cell useful for classifyingcell phenotype, said profile weighted intensity feature determined fromone or more images of the cell, the method comprising the steps of: (a)detecting, by a processor of a computer, a cell and detecting a nucleusof the cell from one or more images of the cell; (b) computing, by theprocessor, a profile image for the cell, wherein pixel intensity of theprofile image is a function of both (i) the closest distance from thepixel to an outer border of the cell and (ii) the closest distance fromthe pixel to an outer border of a nucleus of the cell; and (c)computing, by the processor, a profile weighted intensity feature forthe cell using the profile image of the cell as a weight image, whereinthe profile weighted intensity feature for the cell is a weighted meanintensity of pixels of the cell.
 9. The method of claim 8, wherein theprofile weighted intensity feature for the cell is computed from afiltered image of an acquired input image of the cell using the profileimage of the cell as the weight image.
 10. A method for determining aprofile weighted intensity feature for a cell useful for classifyingcell phenotype, said profile weighted intensity feature determined fromone or more images of the cell, the method comprising the steps of: (a)detecting a plurality of borders of a cell from one or more images ofthe cell; (b) computing a profile image for the cell, wherein pixelintensity of the profile image is a function of the nearest distancesfrom the pixel to each of the plurality of detected borders of the cell;and (c) computing a profile weighted intensity feature for the cellusing the profile image of the cell as a weight image, wherein theprofile weighted intensity feature for the cell is a weighted meanintensity of pixels of the cell.
 11. The method of claim 10, wherein theprofile weighted intensity feature for the cell is computed from afiltered image of an acquired input image of the cell using the profileimage of the cell as the weight image.
 12. An apparatus fordetermination of a profile weighted intensity feature for a cell usefulfor classifying cell phenotype, said profile weighted intensity featuredetermined from one or more images of the cell, the apparatuscomprising: (a) a memory for storing a code defining a set ofinstructions; and (b) a processor for executing the set of instructions,wherein the code comprises a profile weighted intensity moduleconfigured to: (i) detect a cell in an image; (ii) compute a profileimage for the cell, wherein pixel intensity of the profile image is afunction of the closest distance from the pixel to a border of the cell;and (iii) compute a profile weighted intensity feature for the cellusing the profile image of the cell as a weight image, wherein theprofile weighted intensity feature for the cell is a weighted meanintensity of pixels of the cell.
 13. The apparatus of claim 12, whereinthe profile weighted intensity module is configured to classify cellphenotype using the computed profile weighted intensity feature.
 14. Theapparatus of claim 12, wherein the border is the outer border of thecell.
 15. The apparatus of claim 12, wherein the profile image iscomputed such that pixels near the border of the cell are emphasized andpixels a given distance away from the border of the cell aredeemphasized.
 16. The apparatus of claim 12, wherein the profileweighted intensity module is configured to compute a distance image viasliding parabola erosion operation performed on a cell mask image,wherein the profile image is based at least in part on the distanceimage.